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Learning Objectives 


• Objective 1 : Systems Engineering (SE) Processes 
- Defined by NASA PR 7123.1 

• Objective 2: How SE processes are implemented by the 
testing organization 

• Objective 3: Relationship of SE Processes to AS91 00 

• Objective 4: Implications for Quality Management 
Systems for testing 

• Objective 4: Involvement beyond test article inspection 

• Objective 5: Improving effectiveness, and reducing risk 
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Systems Engineering Processes 


Requirements flow down 
from level above 


Realized products 
to level above 



to level below 


from level below 


System design processes 
applied to each work breakdown 
structure model down and 
across system structure 


Product realization processes 
applied to each product 
up and across 
system structure 
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NPR 7123.1 


Product SE Process 


• Most people are familiar with the SE processes 
applied to a product 

• Top level requirements defined by stakeholders 
(e.g. Congress) 

• Agencies and contractors further define details 


Design, acquisition, production, assembly, test, 
delivery ^ 

Technical management 
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Space Shuttle Main Engine 


Evaluation Processes (7&8, “V&V”) 


• In order to prepare a test facility for validation of a 
product, the SE Processes are performed by the 
testing organization on the facility. 

• Process Complexity varies depending on 

- Level (Integrated systems, system, subsystem, 
component, subscale, etc.) 

- Phase (development, V&V, qualification, acceptance) 

- Type of test (flight, “hotfire”, wind tunnel, combined 
environments) 
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MSFC Advanced Engine Test Facility 



Stennis A-1 Test Stand 
MSFC Advanced Engine Test Facility 
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KSC Shuttle Launch Pad 
Vandenburg AFB SLC-6 Space Shuttle 




SE Processes for a Test Facility 


• Customer provides a Test Requirements Document 

• Detailed Test Parameters defined 


• Test engineers break down (decompose) requirements 
into facility systems 

• Test organizations and customers determine whether to 
modify, or build new facilities 

• Testing organization 

- designs, 

- builds hardware 


- prepares infrastructure 

- programs computers 

- establishes and activates support systems (design solution, 
product realization) 

ASQ 
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SE Processes at a Test Facility 


• Test facility systems are installed and connected 
(mechanical, control and instrumentation) 

• Test facility systems’ configuration is verified 

• Test facility systems are tested, checked-out, 
validated 
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MSFC Test Stand 116 
J2-X Gas Generator 


SE Processes at a Test Facility 


Depending on the complexity, the Technical Management Processes 
can be a major activity 

- Technical Planning 

- Management of Requirements 

• Customer, AS9100, OSHA, ISO 14001, technical, policy, etc. 

- Configuration of facility and facility/test article interface 

• Work is often done in parallel 

- Risk Management 

- Data Management 

- Design Reviews, Operational Reviews, 

Readiness Reviews, Hazard Analysis 
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MSFC Dynamic Test Stand 
Space Shuttle Mated Vehicle 
Ground Vibration Test 
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Test Project Process Flow 








Much of the process between kickoff and TRR is done in parallel. 
Technical planningmeetings, reviews and assessments are scheduled as required. 

























Why is this Important? 


• Test facilities have to be safe, reliable 

• Achieved in part by compliance with industry codes and standards 

- Stringent design and quality requirements, configuration control, 
accept/reject criteria (you meet code or you don’t) 

• Mechanical, control and instrumentation systems have to 
communicate, and also communicate with the test article 


• All systems must have configuration control 

• Work authorizing documentation, planning 

• Hazard analysis is based on code 
compliance, configuration control 


Artist Concept: MSFC Test Stand 4693 
Planned for SLS LH2 Tank 



Why is this Jmportant? 


• Multitude of Critical Processes 

- Precision cleaning, field cleaning, 
contamination prevention 

- Calibration 

- Assembly operations; mechanical, 
electrical 

- FOD Control 

- Valve and component servicing 

- Welding, weld inspection and NDE 

KSC Ares-IX Flight Demonstration 

- Transportation and handling 

• Multitude of Safety Considerations 

- Hazardous operations: explosive devices, propellants, high 
pressure, confined spaces, etc. 

- Safety and quality are often inseparable 

• Multitude of Interfaces 
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SE and AS9100 



MSFC Solid Propulsion Test Facility 
M NASA 48” Motor 


SE PROCESS 

AS9100 REQUIREMENT 

Stakeholder Expectations 

Customer Requirements 

T echnical Requirements 
Definition 

Planning of Product Realization 

Logical Decomposition 

Design and Development Input 

Design Solution Definition 

Design and Development Output 

Product Implementation 

Control of Production 

Product Integration 

Control of Production 

Product Verification 

Verification 

Product Validation 

Validation 

Product T ransition 

Control of Work Transfers; Post Delivery Support, 
Preservation of Product 

T echnical Planning 

Planning of Product Realization; Review of Requirements; 
Measurement, Analysis and Improvement 

Requirements 

Management 

Design and Development Planning; Purchasing 

Interface Management 

Configuration Management 

Technical Risk 
Management 

Risk Management 

Configuration 

Configuration Management; Identification and Traceability; 

Management 

Control of Nonconforming Product 

Technical Data 

Control of Documents; Control of Records; Control of Design 

Management 

and Development Changes 

T echnical Assessment 

Design and Development Review 

Decision Analysis 

Measurement, Analysis and Improvement; Analysis of Data 
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Typica[QA£unctions at a Test Site 

• Test Article Inspections 

• Procedure Approval 

• Surveillance of handling and testing 

• Discrepancy Reports 

• Audits 


Super Guppy at MSFC 
Cryogenic Composite Tank 
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How Can QA Support SE/Testing? 

• Develop and manage a relevant, effective QMS* 

• Technical Planning 

- Design, plan, prevention, problem solving 

• Quality Requirements for Procurements 

- Test operations, hardware, software 

• Critical Processes 

- Process improvements, compliance 

• V&V of the Facility 

• Risk Management 

- Identify, mitigate, control 

• Design Reviews & Readiness Assessments 

- Work the issues before TRR 

• *Requires in-depth understanding of SE processes applied to testing and 
various critical processes required to prepare the facility. 

ASQ 
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MSFC Structural Test Facility; Aluminum Lithium Test Article 



Caveats 


• If you wait till the test article is delivered, you 
won’t understand the rest of the activities. 

• Development and “V&V” tests can cost millions of 
dollars and be a significant risk to the programs; 
don’t ignore them. 

• The actual test is often run by computers; don’t 
ignore test data. 

• Don’t write a manufacturing QA Plan for a test 



“Take-Aways” 


• By understanding SE Processes, SE Processes applied 
to testing, and how it all maps to AS9100, the Quality 
Professional can contribute to test projects by: 

- Improving effectiveness of the Quality Management 
System 

- Reducing risk 

- Enhancing safety and sustainability 

- Supporting cost, schedule and 
technical management 

- Improving the auditing process 

- Improving contract requirements 
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